Context-Dependent Control of a Surgical Microscope

ABSTRACT

A parameterization of a control algorithm of a surgical microscope is carried out on the basis of one or more context parameters of an operation. On the basis of the parameterization, the control algorithm then is applied to one or more state indicators associated with a first setting of the surgical microscope in order thus to obtain a second setting of the surgical microscope. By way of example, the parameterization can comprise one or more prioritization operations.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority of German Patent Application No. DE 102020 130 805.1 filed on Nov. 20, 2020, the contents of which areincorporated herein.

TECHNICAL FIELD

Various examples relate to the determination of settings for a surgicalmicroscope. Various examples relate in particular to considering contextparameters of the operation when determining settings.

BACKGROUND

The prior art has disclosed surgical microscopes that offer verydifferent items of information to a user, generally the surgeon, in theeyepiece. By way of example, DE 10203215 A1 describes a surgicalmicroscope that comprises a camera which generates an electronic imagesignal. The image signal is displayed on an electronic eyepiece whichcomprises a corresponding display apparatus for the electronic imagedata. Further items of information may also be output there. A surgicalmicroscope is also known from DE 10 2014 113 935 A1.

Typical surgical microscopes have a multiplicity of possible settings.It can often require much outlay to choose a good setting during theoperation.

EP3593704 is known, inter alia, and discloses an assisting endoscopewhich derives actions on the basis of image processing and a database ofprevious surgeries. A manually created database is used in this case.Such techniques often have restricted flexibility and are thereforesometimes inaccurate.

SUMMARY

Therefore there is a need for techniques which determine a setting for asurgical microscope.

This object is achieved by the features of the independent patentclaims. The features of the dependent patent claims define embodiments.

A setting for a surgical microscope is determined automatically in thevarious examples. In this case, a target conflict between, on the onehand, a suitable setting and, on the other hand, an undisturbed workflowduring the operation can be resolved by virtue of the expected optimalsetting being determined independently and consequently the cognitiveload being reduced.

Further, various examples describe how a further target conflict causedby different requirements of different assistance functionalities can beresolved during the automatic determination of the setting for thesurgical microscope.

A method for controlling a surgical microscope during an operation on apatient comprises the determination of one or more state indicators. Theone or more state indicators are associated with at least one firstsetting of the surgical microscope. The method also comprises theimplementation of a parameterization of a control algorithm of thesurgical microscope on the basis of one or more context parameters ofthe operation. Moreover, the method comprises the application of thecontrol algorithm to the one or more state indicators in order to thusdetermine a second setting of the surgical microscope.

Thus, this means that a control algorithm can initially be suitably setby way of the parameterization and is subsequently used to determine thesecond setting of the surgical microscope.

In general, the second setting of the surgical microscope can differfrom the at least one first setting.

By way of example, the second setting of the surgical microscope couldbe obtained by adapting one or more of the at least one first setting ofthe surgical microscope.

The one or more state indicators can be determined for example on thebasis of a monitoring of the operation and/or on the basis of microscopyimages of the surgical microscope.

In this case, the microscopy images can be captured using the at leastone first setting of the surgical microscope.

As a result of parameterizing the control algorithm it is possible todetermine the second setting in such a way that the latter is adapted tothe best possible extent to a requirement of the surgeon, i.e.,facilitates a best possible assistance of the surgeon during theoperation.

The method can furthermore comprise triggering an application of thesecond setting. To this end it is possible, for example, to provideappropriate control data for the surgical microscope.

The control algorithm can be applied continuously, i.e., it is possibleto continuously determine and apply new settings for the surgicalmicroscope.

In this way there can be a continuous interaction between the surgeonand the control algorithm as the operation progresses because thecontrol algorithm is newly parameterized to the context parametersaltered by actions of the surgeon. Moreover, actions of the surgeon canhave an effect on the state indicators.

There are different variants to carry out the parameterization. By wayof example, the parameterization can comprise carrying out at least oneprioritization operation. The prioritization operation can prioritizecertain processes within the control algorithm over other processes. Theprioritization operation can prioritize certain variables within thecontrol algorithm over other variables.

By way of example, it would be possible to carry out a prioritizationwithin candidate state indicators. These candidate state indicators canbe determined using different metrics on the basis of the microscopyimages of the surgical microscope. Thus, this means that differentproperties or features of the microscopy images can be considered inorder to determine the various candidate state indicators. By way ofexample, different quality features could be taken into account. Asemantic content of the microscopy images could be evaluated differentlydepending on the metric. By way of example, the different metrics couldbe associated with different assistance functionalities for the surgeon.

What can be achieved by the prioritization within the candidate stateindicators is that different properties of the microscopy images areconsidered to a different extent, for example depending on the contextof the operation. This is based on the discovery that differentproperties of the microscopy images may have a different importance tothe surgeon depending on the context of the operation and during thecourse of the operation.

As an alternative or in addition thereto, it would be possible to carryout a prioritization within candidate control models of the controlalgorithm. These candidate control models can be used to ascertainsettings of the surgical microscope on the basis of one or more stateindicators. Thus, expressed differently, such candidate control modelscan translate state indicators into possible settings of the surgicalmicroscope, from which the final second setting can then be selected.

Such techniques are based on the discovery that depending on the contextof the operation different approaches for finding settings of thesurgical microscope can be particularly expedient or can workparticularly accurately.

As an alternative or in addition thereto, it would be possible for theprioritization to be carried out within candidate settings of thesurgical microscope which are obtained by at least one control model ofthe candidate control models.

By way of example, if different variants for the setting of the surgicalmicroscope are present it is possible to choose a particularly suitablesetting, specifically on the basis of the context of the operation, forexample.

The at least one prioritization operation can be carried out to resolveone or more target conflicts. Such a target conflict can becharacterized by different settings of the surgical microscope,depending on the result of the prioritization operation.

Thus, a target conflict might be present if different settings of thesurgical microscope are conceivable. Then, one of these differentsettings can be preferred as a result of the prioritization operation.

In this case, the prioritization operation can be applied near the finalsetting in the processing procedure, that is to say by prioritizationwithin the candidate settings, or else be applied upstream within theprocessing procedure, for example by prioritization within the candidatecontrol models or even further upstream by prioritization withincandidate state indicators.

The at least one prioritization operation may comprise filtering and/orweighting the candidate state indicators and/or the candidate controlmodels and/or the candidate settings.

By way of example, filtering may comprise discarding a candidate stateindicator or a candidate control model or a candidate setting, withhowever a respectively different candidate being kept.

The weighting can keep a plurality of appropriate candidates butconsider these to different extents in the subsequent processingprocedure. In this way a flexible prioritization can attain particularlyaccurate results. It would be possible for the one or more contextparameters to comprise one or more of the following elements: a surgicaldevice, for example a surgical tool; a region of interest of theoperation; a user stipulation; a phase of the operation; a type ofoperation; activities of the operation; planning data for the operation;and/or at least one surgeon involved in the operation. There can be aparticularly comprehensive and flexible parameterization by way of suchdifferent elements which are taken into account in conjunction with thecontext.

The one or more context parameters can also comprise a prediction for afuture progress of the operation.

As an alternative or in addition thereto it would be possible for theone or more context parameters to comprise a previous, preceding courseof the operation.

The one or more context parameters can also describe a current actualstate of the operation.

By way of example, the further course of the operation could also bepredicted from a previous course of the operation and the current stateof the operation.

By way of example, such a prediction could take account of the fact thatthe operation has already progressed during the time taken to determineand possibly apply the second setting. This delay can then be consideredby the prediction of the course of the operation within the scope of thecontext parameter.

It would be possible for at least one context parameter of the one ormore context parameters to be determined on the basis of a monitoring ofthe behaviour of the surgeon involved in the operation.

By way of example, to this end there could be an application of surroundsensors, voice recognition could be used, the motion behaviour of thesurgeon could be taken into account and/or the input of the surgeon viaa human-machine-interface could be taken into account, to name but a fewexamples.

In particular, such techniques can facilitate an individualdetermination of the context depending on the surgeon, and hence canfacilitate a targeted parameterization of the control algorithm.

The monitoring of the operation which is taken into account whendetermining one or more state parameters could for example relate to anumber and/or an arrangement of one or more regions of interest of theoperation in the patient. The regions of interest might be particularlyindicative for the state of the operation. Specifically, the regions ofinterest can denote the regions focussed on by the surgeon. By way ofexample, a region of interest of the operation can refer to a regionwithin the situs or the entire situs, which is of interest in anappropriate context of the operation.

This allows the state of the operation to be evaluated well in relationto the regions of interest. Accordingly, it would then be conceivablefor an associated state parameter to evaluate, e.g., the visibility orrecognizability of features within the region of interest in amicroscopy image of the surgical microscope. As an alternative or inaddition thereto the operation could be monitored on the basis of aspecified model of the operation, which may comprise user-specificvariants, for example. By way of example, this means that the specificcourse of the operation can be compared to a template provided by themodel. By way of example, the regions of interest, for instanceaccording to the phase of the operation, can be determined by the model.By way of example, it would be possible to monitor deviations betweenthe specific operation and a template by the model and a region ofinterest or, more generally, a state indicator could be determined onthe basis of such deviations. The parameterization itself could becarried out on the basis of an appropriate algorithm. A correspondingalgorithm could be machine-learned. By way of example, reinforcementlearning could be used. Such reinforcement learning can be trained onthe basis of feedback which comprises a difference between the secondsetting of the surgical microscope determined by the control algorithmand a manually selected setting of the surgical microscope.

In this way, the parameterization could implement the stipulations ofthe corresponding surgeon with increasing accuracy as a result of acontinuous interaction between the surgeon and the parameterizationalgorithm. A separate training phase can be dispensed with.

By way of example, the one or more state indicators could be determinedon the basis of metrics which are applied to the microscopy images.

Different metrics are conceivable. By way of example, the metrics couldevaluate one or more of the following properties of microscopy images:an image quality; a semantic content of the microscopy images; and/or avisibility of regions of interest in the microscopy images.

By way of example, the image quality could denote a brightness and/or acontrast and/or a colour space of the microscopy images. In thisrespect, the image quality can be independent of the semantic content ofthe microscopy images, i.e., for example, independent of the visibilityof a certain region of interest.

By way of example, a check could be carried out as to whetherreflections are present within a certain region of interest.

By way of example, a check could be carried out as to whether aparticularly low contrast is present within a certain region ofinterest.

By way of example, a check could be carried out as to whether there is asaturation in a certain region of interest of the pixel values of thevarious pixels of the microscopy images which image the correspondingregion of interest.

By way of example, different state indicators could be associated withdifferent assistance functionalities. In turn, these assistancefunctionalities can be associated with different metrics.

The metrics may cause an above-described target conflict. This may bethe result of the different metrics assessing the same properties of themicroscopy images contrariwise.

The monitoring of the operation can relate to at least one of thefollowing: a course of the operation; an actual state of the operation;and/or a progress of the operation in relation to a target state. By wayof example, it would be conceivable for a specified model of theoperation to be taken into account to this end. By way of example, theprogress of the operation can describe a degree of fulfilment of one ormore targets defined by the specified model.

By monitoring the operation on the basis of such criteria it is possiblein particular to monitor a deviation from certain stipulations and oneor more state indicators can be determined on the basis thereof. Then,these are particularly meaningful in order to bring about acorresponding adjustment of the setting of the surgical microscope whichfacilitates an attainment of the stipulations.

The control algorithm could comprise one or more control models. Such acontrol model is configured to determine a candidate setting of thesurgical microscope on the basis of at least one corresponding stateindicator.

By way of example, machine-learned algorithms can be used as controlmodels. Machine-learned algorithms of the various control models can betrained separately.

By way of example, this can allow the provision of a modular controlalgorithm which can be flexibly extended. Certain control models can beinterchanged. New control models can be added. Old control models can beremoved.

A computer program or a computer program product or a computer-readablestorage medium comprises program code. The latter can be loaded andexecuted by a processor. The processor executing the program code causesthe processor to carry out a method for controlling a surgicalmicroscope during an operation on a patient. The method comprises thedetermination of one or more state indicators. The one or more stateindicators are associated with at least one first setting of thesurgical microscope. The method also comprises the implementation of aparameterization of a control algorithm of the surgical microscope onthe basis of one or more context parameters of the operation. Moreover,the method comprises the application of the control algorithm to the oneor more state indicators in order to thus determine a second setting ofthe surgical microscope.

A device comprises a processor. The processor is configured to determineone or more state indicators on the basis of a monitoring of anoperation and further on the basis of microscopy images of a surgicalmicroscope which are captured using at least one first setting of thesurgical microscope. The one or more state indicators are associatedwith the at least one first setting of the surgical microscope.Moreover, the processor is configured to carry out a parameterization ofa control algorithm of the surgical microscope on the basis of one ormore context parameters of the operation. Further, the processor isconfigured to apply the control algorithm to the one or more stateindicators in order to thus determine a second setting of the surgicalmicroscope. This use of the control algorithm is based on theparameterization.

The features set out above and features that are described below may beused not only in the corresponding combinations explicitly set out, butalso in further combinations or in isolation, without departing from thescope of protection of the present disclosure.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 schematically illustrates a surgical microscope as per variousexamples.

FIG. 2 schematically illustrates a region of interest of an operationand the relative positioning of the surgical microscope in relation tothe region of interest according to various examples.

FIG. 3 schematically illustrates a device for data processing inaccordance with various examples.

FIG. 4 schematically illustrates a control algorithm as per variousexamples.

FIG. 5 is a flowchart of one exemplary method.

FIG. 6 schematically illustrates a parameterization of a controlalgorithm as per various examples.

FIG. 7 schematically illustrates a parameterization of a controlalgorithm as per various examples.

FIG. 8 schematically illustrates a parameterization of a controlalgorithm as per various examples.

FIG. 9 schematically illustrates a parameterization of a controlalgorithm as per various examples.

FIG. 10 illustrates an algorithmic implementation of a control algorithmas per various examples.

DETAILED DESCRIPTION

The properties, features and advantages of this disclosure describedabove and the way in which they are achieved will become clearer andmore clearly understood in association with the following description ofthe exemplary embodiments which are explained in greater detail inassociation with the drawings.

The present disclosure is explained in greater detail below on the basisof preferred embodiments with reference to the drawings. In the figures,identical reference signs denote identical or similar elements. Thefigures are schematic representations of various embodiments of thedisclosure. Elements illustrated in the figures are not necessarilyillustrated as true to scale. Rather, the various elements illustratedin the figures are rendered in such a way that their function andgeneral purpose become comprehensible to the person skilled in the art.Connections and couplings between functional units and elements asillustrated in the figures can also be implemented as an indirectconnection or coupling. A connection or coupling can be implemented in awired or wireless manner. Functional units can be implemented ashardware, software or a combination of hardware and software.

Various examples of the disclosure relate to the determination of asetting of a surgical microscope during the operation. Differentsettings of the surgical microscope can be determined in the variousexamples described herein. By way of example, it would be conceivablefor relative positioning, i.e., a distance and/or orientation (pose), ofthe surgical microscope to be determined in relation to the patientundergoing the operation. As an alternative or in addition thereto, itwould also be possible to determine settings of the optical system inthe surgical microscope, for example a magnification (zoom), anillumination intensity and/or a contrast. It would also be possible todetermine a mode of operation, for example the use of indirect or directillumination, or illumination with light at a certain wavelength. Afluorescence mode could be activated. Video settings could be used.

Various examples are based on the discovery that a manual determinationand application of settings—for instance manual repositioning of thesurgical microscope—may represent an additional cognitive and mentalload for the executing surgeon during the operation. By way of example,manual repositioning requires a “free hand”, and so the surgeon must putdown the surgical instruments or await a pause for a change of thesurgical instruments. If the surgical instruments are put down in adedicated step manual repositioning leads to an interruption of theoperation. If the surgeon waits for a suitable window of opportunity forthe repositioning there is the risk of the surgeon at least in partcarrying out the operation with sub-optimal settings for the surgicalmicroscope. Accordingly, it is possible to automatically determine thesettings of the surgical microscope in accordance with the variousexamples described herein.

Moreover, the settings of the surgical microscope can be appliedautomatically following the determination.

A reference implementation for automatic determination of the setting ofthe surgical microscope would be, e.g., tracking the head of thesurgeon. Then, the surgical microscope can follow the head movements.Various examples are based on the discovery that such an automaticcontrol method is faced with the following applicative problem inparticular: During surgery, the physician sets the microscope such thatthe visualization meets their demands. The physician sets the microscopeon the basis of their experience; in the process the physiciansubconsciously makes compromises, as should be elucidated using theexample of the choice of zoom level: On the one hand, the microscopeshould be set so that the surgeon can clearly see details of a vessel,for example. This would require a high magnification of the vessel. Onthe other hand, the surgeon would like to be able to work “quickly” inthe situs, wherein a zoom level that is too high would bedisadvantageous. A further example consists of the surgeon liking to seethe surgical instruments situated in the situs centrally in the image.Since the instruments are at different locations on the image and bothinstruments cannot be in the centre of the image at the same time, thesurgeon must also find a compromise in this case. The surgeon solvesthis target conflict subconsciously.

According to various examples it is possible to automatically solve sucha target conflict between different possible settings of the surgicalmicroscope. In some examples, the subconscious setting of a compromisebetween various target conflicts, learned by the surgeon, can beimplemented algorithmically.

Such techniques are based on the discovery that conventional manualsetting of the surgical microscope often has a target conflict betweenoptimal visualization and unimpeded workflow. Frequent repositioning isrequired to attain an average high visualization quality but frequentrepositioning also increases the cognitive load or number of workflowinterruptions.

In the various examples described herein, a context of the operation isconsidered in conjunction with the determination of settings of thesurgical microscope. In this case, the context of the operation can bedescribed by one or more context parameters. The context of theoperation can describe a situation in which the operation is found. As ageneral rule, the context can comprise all objects, regions ofinterests, actors and actions in the operating theatre. By way ofexample, the context can be captured by motion sensors, imageprocessing, speech processing, etc. The context or a correspondingcontext parameter can be determined by the analysis of the situs and useof the surgical microscope. The context can also comprise anticipatedfuture actions. The current context can therefore be used to predictfuture objects, actors and actions during the operation. Example: Thenext operation phase can be predicted on the basis of video data and anoperation phase model. Further example: The region of interest in whichthe surgeon wants to primarily work in future (derived, e.g., from thetime spent).

Very different context parameters can be taken into account as a generalrule. Some of these context parameters are listed below in Table 1.

TABLE 1 Brief description Exemplary details I Surgical tool By way ofexample, a context parameter could be used to indicate which surgicaltool is currently being used. What surgical tool has already been usedor will be used in future during the respective operation could also beindicated. By way of example, the context parameter could specify an IDnumber and/or a type number of the respective surgical tool. Theutilized surgical instruments or surgical tool can be taken intoaccount. II Region of interest By way of example, it would be possibleto in the patient specify which region of interest or regions ofinterest are relevant to the surgeon, either currently or in futureduring the operation. As a general rule a region of interest candescribe, for example, an anatomical region or an anatomical feature inthe patient which requires the attention of the surgeon in the contextof the operation. By way of example, the region of interest could denotethe region of the surgical intervention (situs), or a portion thereof.The context parameter could specify the region of interest inparameterized form, that is to say for example “2 cm around thetooltip”. The context parameter could already annotate the region ofinterest in a microscopy image of the surgical microscope. III Userstipulation The user could manually determine one or more contextparameters - for example pre- surgery. By way of example, this wouldallow user preferences to be represented. By way of example, historicuser data and/or use data could be taken into account. It would also beconceivable for user feedback to be provided at the runtime. User inputscan be taken into account. The user stipulation could be stored in auser profile, for example. IV Phase of the An operation can typically besubdivided into a operation plurality of sections or phases. Thedifferent phases of an operation may be associated with differentregions of interest, for example. The different phases of an operationcan also be associated with different medical devices or surgicalinstruments (tools). V Type of operation By way of example, the type ofoperation can describe a basic form of the surgical intervention. By wayof example, the following would be conceivable types of operation:spinal column; microbiological intervention; head; brain; eye; etc. VISurgeon A certain type of operation can follow different coursesdepending on the surgeon. By way of example, different surgeons havedifferent preferences in relation to the region of interest in thepatient which should be represented, for example, in microscopy imageson the surgical microscope. Different surgeons may also implementdifferent types of intervention. It would be possible in some examplesto dynamically monitor the behaviour of the surgeon during theoperation. In this way it may then be possible to react even to changesin the course of the operation which are dynamically prompted by thesurgeon during surgery. VI Analysis of the By way of example, it wouldbe conceivable to situs determine one or more characteristics of thesitus of the operation by means of an image analysis of image datacaptured pre-surgery and/or during surgery. By way of example, it wouldbe possible to determine placement and/or extent. By way of example,imaging techniques such as magnetic resonance imaging or computedtomography could be used to this end. VII Activities during By way ofexample, it would be conceivable to the operation monitor certainlandmarks of the course of the operation. By way of example, it ispossible to monitor certain milestones in a model of the operation beingreached. This could correspond to a phase of the operation, cf. ExampleIV.VIII VIII Movement pattern By way of example, movement patterns ofthe of the surgical surgical instruments could be monitored, forinstruments example by object recognition in the corresponding imagedata or by way of active tracking. IX Randomization Sometimes it is alsopossible to take account of a randomized component in conjunction withthe context parameter. Randomization is traced back to the concept of analgorithm recognizing that it understands the context onlyinsufficiently and then nevertheless is able to provide an acceptable(albeit non-optimal) decision basis for determining the setting by wayof randomization. X Planning data By way of example, planning datacaptured for the operation pre-surgery or specified planning data couldbe taken into account. These can, but need not be available as userstipulations.

Table 1:Examples of different context parameters that can be taken intoaccount in conjunction with the parameterization of the controlalgorithm. As a general rule, the context parameters can be defined inconjunction with an actual state of the operation, meaning for examplethat these describe a current phase of the operation or a current regionof interest in the patient. However, as an alternative or in additionthereto it would also be conceivable for the one or more contextparameters to comprise a prediction about the future progress of theoperation. Thus, it would be conceivable for a context parameter tospecify a future region of interest. This can facilitate an anticipatoryadjustment of the setting of the surgical microscope.

Depending on the information content of the context parameters it ispossible to use different techniques for determining the contextparameters. Some context parameters can be specified statically, i.e.,read from a memory. Other context parameters can be determined, e.g.,dynamically, for instance during surgery. By way of example, differentsensor data could be used to this end. It is possible to use acorresponding algorithm for determining the one or more contextparameters, for example a machine-learned method.

According to various examples one or more context parameters—cf. Table1—of the operation are used in conjunction with determining a setting ofthe surgical microscope. In this case, the one or more contextparameters of the operation are used in conjunction with theparameterization of a control algorithm which is configured to determinethe setting of the surgical microscope. In this case, parameterizationmeans that one or more parameters of the control algorithm are setdepending on the one or more context parameters, for example this meansthat corresponding processing boundary conditions are pre-setaccordingly before, specifically, an output is calculated. A possibleexample of the parameterization would be the prioritization betweendifferent settings and/or results of the control algorithm. By way ofexample there could be a selection between a plurality ofsub-algorithms. Inputs could be taken into account in filtered orweighted fashion. It would be possible for different calculations to besubsequently carried out, depending on the parameterization, whenascertaining an output on the basis of an input. Taking into account theone or more context parameters within the scope of the parameterizationcan in particular differ from taking into account one or more inputswhen determining an output by the control algorithm. The inputs of thecontrol algorithm correspond to variables which directly influence theresult, i.e., the output, of the control algorithm. Then again, theparameterization corresponds to a basic configuration of the controlalgorithm.

The implementation of the parameterization can be carried out in turn byan algorithm (algorithm for the parameterization). The algorithm for theparameterization can receive one or more context parameters as an inputand can carry out the parameterization of the control algorithm on thebasis thereof. By way of example, the algorithm for the parameterizationcould use a machine-learned method. In particular, reinforcementlearning could be used. In this case, a loss function could be definedon the basis of a difference between the setting for the surgicalmicroscope as determined by the control algorithm and a setting of thesurgical microscope following a manual readjustment by the surgeon. Thealgorithm for the parameterization can be trained on the basis of thisloss function, wherein the control algorithm itself may remain fixed,however.

The control algorithm is applied to one or more state indicators in thevarious examples. This means that the one or more state indicatorsrepresent the input of the control algorithm.

In general, the state indicators could describe features of theoperation. In this case, the state indicators can describe a stateassociated with at least one first setting of the surgical microscope,wherein the control algorithm then determines a second setting of thesurgical microscope that differs from the at least one first setting. Byway of example, the second setting can represent an adjustment of one ormore of the at least one first setting. By way of example, the at leastone first setting could currently be in use and/or comprise one or moresettings used earlier.

As a general rule, the state indicators can characterize individualfeatures of the visualization quality. Exemplary state indicators wouldbe, e.g.: relative position of regions of interest in the field of viewof the microscope (viewing angle of the cameras); concealment of regionsof interest on surgical instruments or the situs by surgical instrumentsor tissue of the situs; and/or illumination of the situs by comparingthe position of a region of interest with an optical axis.

In general terms it is possible for the one or more state indicators tobe determined on the basis of microscopy images of the surgicalmicroscope. Thus, it would be conceivable for the one or more stateindicators to be determined on the basis of microscopy images from thesurgical microscope, using one or more specified metrics.

The one or more specified metrics can thus map the microscopy images onthe set of state indicators. As a general rule, very differentproperties of the microscopy images can be taken into account in thiscase when determining the state indicators. By way of example, it wouldbe conceivable that an image quality of the microscopy images isevaluated. To a first approximation, the image quality can be evaluatedindependently of the semantic content of the microscopy images in thiscase, for example independently of the represented anatomical featuresor other objects imaged of the microscopy images. By way of example, theimage quality could relate to a brightness and/or a contrast of themicroscopy images. However, as an alternative or in addition thereto itwould also be possible for at least one of the one or more metrics toalso evaluate a semantic content of the images. By way of example, thevisibility of certain anatomical features or, phrased more generally, ofregions of interest in the patient could be evaluated. By way ofexample, the visibility could be impaired by concealment, for instanceby surgical instruments, or by reflections.

In this case the state indicators can quantitatively describe suchfeatures, as described above. It would be conceivable for the stateindicators to each be associated with a corresponding targetstipulation, and to each describe the degree of fulfilment of thistarget stipulation. By way of example, a state indicator could count thenumber of reflections and there could be a threshold for the number ofreflections as a target stipulation.

Depending on the form of the metric it is possible to use very differenttechniques for the algorithmic implementation of a correspondingalgorithm for determining the state indicators. By way of example, ifthe semantic content of microscopy images is evaluated, an objectrecognition could be used to determine the state indicators. If theimage quality, for example the brightness or the contrast, is assessedit would be possible to carry out a histogram analysis of brightnessvalues of the microscopy images. A corresponding (sub-)algorithm can bepart of the control algorithm.

By way of example, it would be conceivable that the various stateindicators are associated with different assistance functionalities forthe surgeon. It is then possible for state indicators to be obtained bydifferent metrics which are applied to the microscopy images of thesurgical microscope, wherein the different metrics are associated withthe different assistance functionalities. Examples of assistancefunctionalities would be, e.g.: homogeneous illumination of the situs;visibility of a certain region of interest; suppression of reflectionson the image; etc.

By virtue of using different metrics it is possible to assess the sameproperties of the microscopy images differently. This is due to the factthat the visibility of a certain region of interest may be important fora first assistance functionality for example, with the visibility ofthis same defined region of interest possibly being unimportant to asecond assistance functionality. Thus, this means that the differentmetrics (for instance of the various assistance functionalities) assessthe same properties of the microscopy images contrariwise. Accordinglythere is a target conflict which can be expressed by the correspondingstate indicators associated with the various assistance functionalities.

In general, target conflict can mean that a plurality of targetstipulations—e.g., in conjunction with different assistancefunctionalities or more generally different metrics for evaluatingmicroscopy images in the form of the state indicators—should besimultaneously fulfilled to the best possible extent. In this context,target conflict means that the target stipulations are contrary to oneanother—for example comprehensive overview when working (lowmagnification of the situs) versus a detailed representation of thevessels in the situs (high magnification). Since the surgical microscopeonly has one parameter for the zoom setting the latter cannotsimultaneously fulfil both target stipulations.

As a general rule it would be possible in this case for the stateindicators to be determined dynamically. This means that the stateindicators are determined intermittently—e.g., also in real time, i.e.,with a cycle frequency of e.g. 20 Hz or faster. Thus, it is possible forthe state operators to also be determined on the basis of a monitoringof the operation. In particular, it would be conceivable that anassociated target stipulation also depends on the monitoring of theoperation. This means that it is possible to determine whether themicroscopy images captured by a current or a preceding setting of thesurgical microscope are appropriate in conjunction with, for instance, acurrent state or course of the operation. This means that differentstate indicators can be obtained, optionally dynamically on the basis ofthe course of the operation, and so in each case different results forthe setting of the surgical microscope are obtained dynamically. In thisway, it is possible to react to the respective situation of theoperation and a respective setting can be selected.

The monitoring of the operation can denote, e.g., a course of theoperations, an actual state of the operation and/or a progress of theoperation in relation to a target state. The monitoring of the operationcould also relate to a prediction for a future state of the operation.

In this case, the monitoring of the operation can relate to, e.g., anumber and/or arrangement of one or more regions of interest of theoperation within the patient. As a general rule, a region of interestcan be the general designation for a region in the situs which is ofinterest to the surgeon. By way of example, regions of interest caninclude anatomical features (tumour, nerves, . . . ), events in thesitus such as haemorrhaging, navigation data, planned trajectories,fibre tracts, . . . . However, regions of interest can also be“attached” to the tip of a surgical instrument and then moved with thesurgical instrument.

Then, it is possible—depending on the state of the operation—fordifferent regions of interest to be relevant to the surgeon. It would bepossible for the monitoring of the operation to be based on a specifiedmodel of the operation. The model of the operation can specify one ormore intended courses of the operation. The regions of interest can alsobe defined in conjunction with such intended courses. It is conceivablefor such a model to comprise user-specific variants. As a general rule,different courses may be stored in a model for the operation.

FIG. 1 schematically shows a surgical microscope 801 for surgery. In theillustrated example, the surgical microscope 801 comprises an eyepiece803. Through the eyepiece 803, the surgeon can observe magnified imagesof an object which is situated in a field of view 804 of the surgicalmicroscope 801. In the illustrated example, this is a patient 805 lyingon a patient couch.

As an alternative or in addition to an optical eyepiece, provision couldalso be made for a camera which transmits images to a screen (digitalsurgical microscope).

An operating device 808 is also provided as a human-machine interface;by way of example, it can be embodied as a handle or a foot switch. Itis a handle in the illustrated embodiment of FIG. 1. The operatingdevice 808 allows the eyepiece 803, which is fastened to crossbeams 850,to be moved. Motors can be provided in order to automatically carry outthe movement on the basis of control data, in accordance with acorresponding setting of the surgical microscope. The motors could alsoassist the movement prompted by the handle 808.

Further, a control device 809 is provided for the surgical microscope801 and controls the operation of the combination microscope and thedisplay of images and additional information and data in the eyepiece803. The control device 809 can interact with the surgeon. By way ofexample, the control device 809 could alter a setting of the surgicalmicroscope on the basis of appropriate control data. To this end, one ormore actuators could be controlled, for instance to move the crossbeam,to change an optical unit, etc. The setting can also comprise thedigital post-processing of sensor data. The setting could also relate todata capture parameters, for instance for captured digital images. Itwould also be possible to switch between different image sources orimaging modes, depending on the setting.

The surgical microscope 801 can also comprise one or more furthersensors 860, for instance a motion sensor or a thermal imaging camera ora microphone or a surround camera, etc. Such further sensors 860 canalso be operated differently depending on the setting of the surgicalmicroscope.

In neurosurgery, surgical microscopes are used to visualize low-lyingstructures in narrow cavities. Depending on the type of operation it isnecessary for the surgical microscope to adopt a new viewing directionrelative to the situs approximately every minute since a new concealmentsituation arises, by way of example, on account of altered positions ofthe instruments. In surgical microscopes that are currently commerciallyavailable it is necessary to this end for the surgeon to reposition thesystem manually, i.e., they clasp, e.g., a handle attached to themicroscope side and guide the system into a new pose (position &orientation of the microscope).

FIG. 2 illustrates an exemplary positioning of an optical unit 806 ofthe surgical microscope 801 in relation to a situs 53 within theskullcap 54 of the patient. Moreover, surgical instruments 51, 52 areillustrated. In the various examples described herein it is possible forthe relative positioning of the surgical microscope 801—for instance ofthe light microscope 806, in particular—in relation to the patient to bedetermined within the scope of setting the surgical microscope 801 andthis can optionally also be adapted automatically, for example by virtueof a corresponding stand comprising mechanical actuators. The relativepositioning can comprise a distance and/or orientation (pose).

FIG. 3 schematically illustrates a device 90 that can be used for thedata processing in the various examples described herein. By way ofexample, the device 90 could be a PC or a cloud server. The device 90comprises a processor unit 91 and a non-volatile memory 92. Theprocessor unit 91 can load program code from the non-volatile memory 92and execute said code. This causes the processor unit 91 to carry outtechniques according to the examples described herein, for exampleparameterizing a control algorithm which is configured to determine asetting of the surgical microscope—for example a second settingproceeding from one or more first settings; determining one or morestate indicators associated with a current setting of the surgicalmicroscope, for example on the basis of microscopy images from thesurgical microscope and optionally using a suitable metric; etc.

FIG. 4 illustrates aspects in connection with the control algorithm 130.As an output the control algorithm 130 provides a setting 125 of thesurgical microscope. By way of example, the setting could be describedby way of appropriate control data which are subsequently transmitted tothe control device 809. The setting of the surgical microscope 801 couldbe implemented automatically on the basis of the control data.

This setting 125 can also be referred to as second setting if it isdetermined proceeding from at least one first setting.

As an input, the control algorithm 130 in the example of FIG. 4 receivesa plurality of state indicators 121-123 which are associated with the atleast one first setting of the surgical microscope 801, that is to say,for example, with the current setting of the surgical microscope orelse, for example, with one or more earlier settings. As a general rule,the control algorithm 130 could also receive only a single stateindicator or two or more state indicators as an input. The controlalgorithm 130 could even determine the state indicators 121-123 itselfin some examples.

The state indicators 121-123 can be determined on the basis ofmicroscopy images of the surgical microscope (captured with a respectivefirst setting) and/or further on the basis of a monitoring of theoperation, that is to say, for example, on the basis of correspondingstate data. By way of example, different state indicators 121-123 can beassociated with different metrics which are applied to the microscopyimages. The different metrics can be associated with differentassistance functionalities, for example. The different state indicators121-123 can describe how well the respectively associated first settingfulfils a certain target stipulation described by the respective metric.By way of example, a certain assistance functionality may demandvisibility of a certain region of interest; if the corresponding regionof interest is not visible or only visible to a restricted extent in themicroscopy images, the respective state indicator can indicate a poorcorrespondence with this stipulation.

FIG. 4 further illustrates that the control algorithm 130 isparameterized on the basis of a context parameter 129. This means thatone or more parameters of the control algorithm 130 are adapteddepending on the value of the context parameter 129. By way of example,a prioritization could take place. In turn, the context parameters couldbe associated with the at least one first setting of the surgicalmicroscope. This means that the context parameter 129 can describe thecontext of the operation in the case of the respectively activated firstsetting.

As a general rule, more than a single context parameter 129 could alsobe used for the parameterization. Examples of context parameters weredescribed above in conjunction with Table 1.

What can be achieved by virtue of the control algorithm 130 beingparameterized on the basis of the context parameter 129 is that thesetting 125 can be determined particularly reliably and dynamically. Inthis way it is possible, in particular, to resolve target conflictswhich might result from state indicators 121-123 associated withdifferent metrics and possibly different assistance functionalities.Phrased more generally, it is possible to resolve target conflicts thatmight result from different stipulations in relation to the setting 125.

FIG. 5 is a flowchart of an exemplary method. By way of example, themethod of FIG. 5 could be carried out by the device 90, for example bythe processor unit 91, on the basis of program code that is loaded fromthe memory 92. Optional boxes are illustrated using dashed lines.

Initially one or more state indicators are determined in Box 3010. Thisis based on the operation being monitored—i.e., for example, on thebasis of one or more state data—and further on microscopy images of thesurgical microscope. By way of example, use could be made of one or moremetrics which—depending on the state of the operation—assess themicroscopy images, for example in respect of fulfilling one or morestipulations.

By way of example, the operation could be monitored in respect of one ormore regions of interest. It would be possible to carry out a continuousidentification of regions of interest. Firstly, a region of interest canbe determined by sensor signals, such as tool movements, for example.Secondly, there is the option of extracting regions of interest by wayof image processing approaches. Thus, tool positions/movements,haemorrhaging, blood vessels, nerves or tumour regions, for example, canbe extracted from image data. It would also be possible to defineregions of interest as regions with increased activity, for example byway of tool tips. Moreover, there is the option of extracting regions ofinterest by including a surgical navigation system. By way of example,the planned trajectory and the current trajectory to the region ofinterest can originate from a navigation system.

In this case, the one or more state indicators can be associated with atleast one first setting of the surgical microscope.

One or more context parameters are optionally determined in Box 3015. Ifthe context parameters are not determined dynamically it would bepossible for these to be predefined pre-surgery.

Examples of context parameters were elucidated above in conjunction withTable 1.

Then, the control algorithm is parameterized in Box 3020. This isimplemented on the basis of the one or more context parameters in Box3015.

As a general rule, a further algorithm (algorithm for theparameterization) could be used for the parameterization. By way ofexample, the latter could be machine-learned. By way of example, usecould be made of an artificial neural network. The machine-learnedalgorithm could be trained in a training phase. The training phase couldbe separate from the actual inference phase, which occurs during theoperation. However, the use of reinforcement learning would also beconceivable. In this case, a comparison could be made as to whether thesecond setting of the surgical microscope ascertained by the controlalgorithm is changed to a manually selected setting of the surgicalmicroscope or whether it is adopted by the surgeon. A loss function forthe training can be defined on the basis of a deviation between themanually selected setting and the setting determined by the controlalgorithm.

Then, the control algorithm can be applied in Box 3025 in order toobtain the second setting of the surgical microscope.

It would be possible for this second setting of the surgical microscopeto then also be applied, for example by virtue of appropriate controldata being transmitted to the control device of the surgical microscope.This corresponds to an action to change setting parameters of thesurgical microscope.

The actions of Box 3010-3025 can optionally be repeated multiple timesin order thus to determine dynamic settings for the surgical microscope.This could be implemented in real time, for example, that is to say witha cycle frequency that is no slower than, e.g., 25 Hz, for example.

Next, examples of the parameterization of the control algorithms 130 inBox 3020 are described. In particular, the parameterization can comprisethe prioritization of parameters of the control algorithm 130. This isexplained in detail below.

In this case, the prioritization can be implemented in different wayswithin the sequence of logical operations of the control algorithm 130,specifically by the prioritization of the available state indicators,control models for ascertaining settings of the surgical microscopeand/or between different candidate settings.

Phrased in general, the prioritization can thus mean that certain stateindicators, control models and/or settings are selected or weighted onthe basis of one or more context parameters.

The parameterization of the control algorithm can thus comprise theimplementation of at least one prioritization operation. A few examplesfor these prioritization operations are described below in Table 2.

TABLE 2 Examples of prioritization operations Exemplary description IPrioritization By way of example, if a plurality of candidate withincandidate state indicators are present (cf., e.g., FIG. 4 stateindicators where a plurality of candidate state indicators 121-123 arepresent) then one or more state indicators can be selected from this setand can be processed further in the control algorithm. This means thatan input layer of the control algorithm can carry out filtering in thecontext of the candidate state indicators. Non-utilized candidate stateindicators can then be discarded. As an example, a concealment of aregion of interest, for example, could be problematic in a certaincontext of the operation (e.g., if the bipolar coagulator opticallyconceals the area to be coagulated) while concealment may beunproblematic in a different context (e.g., if the aspirator concealsthe bipolar only briefly). If it is recognized on the basis of a contextparameter that concealment critically reduces the visualization qualitythen a state indicator which quantifies the “concealment” is prioritizedand used to determine the setting. By way of example, this isimplemented by virtue of, e.g., a pose which reduces the concealmentthat is problematic for the visualization quality being ascertained whendetermining the setting. Thus, this means that the activation of thecorresponding setting of the surgical microscope improves thecorresponding state indicator which quantifies the “concealment” (i.e.,brings it better into correspondence with a corresponding targetstipulation). Other state indicators might not be taken into account oronly taken into account to a subordinate extent. On the basis of thecurrent course of the operation and a corresponding context parameter, acurrent region of interest is determined as being located away from thecentre of the field of view of the surgical microscope and hence thereno longer is optimal illumination of the region of interest. Since thisregion of interest has already been detected for a specified time periodthe anticipated future action is that the surgeon wishes to continue towork in this region of interest. A state indicator which quantifies thecurrently incorrect illumination is therefore prioritized and in thisway an adjusted illumination is determined as an action as a setting ofthe surgical microscope. Other state indicators might not be taken intoaccount or only taken into account to a subordinate extent. IIPrioritization By way of example, the control algorithm can withincandidate comprise different candidate control models control modelswhich can be used to ascertain settings of the surgical microscope onthe basis of at least one state indicator and optionally an associatedtarget stipulation. By way of example, different candidate controlmodels can comprise different approaches for generating appropriatecontrol data. Different degrees of freedom of the surgical microscopecan be addressed. It would be possible to then select one or more ofthese candidate control models. III Prioritization It would beconceivable that a plurality of within candidate candidate settings - ofone or more candidate settings control models - are received. It is thenpossible to select the setting output at the end (also referred to as“second setting” above) from the corresponding set. Thus, it is possiblefor the control algorithm to carry out appropriate filtering between thecandidate settings.

Table 2:Examples of prioritization operations. It is possible for theprioritization to comprise a combination of such prioritizationoperations and/or further prioritization operations. The prioritizationoperation can be carried out to resolve at least one target conflict.The target conflict can be characterized by different settings of thesurgical microscope, depending on the result of the prioritizationoperation. This means that—depending on the result of theprioritization—a setting is obtained for the surgical microscope 801 ineach case and different results of the prioritization are associatedwith different settings. These can be formed contrariwise, i.e., atarget conflict is present. Thus, this means that the prioritizationoperation resolves the target conflict to the benefit of the ultimatelydetermined setting. This is achieved by virtue of the candidate stateindicators and/or candidate control models and/or candidate settingsbeing taken into account in weighted fashion and/or being filtered(i.e., some elements are discarded).

The various examples of prioritization operations as per Table 2 aredescribed in more detail below in conjunction with FIG. 6, FIG. 7, FIG.8 and FIG. 9.

FIG. 6 illustrates aspects in connection with the control algorithm 130.As an input, the control algorithm 130 receives a plurality of candidatestate indicators 320 (these can correspond to the state indicators121-123 as per FIG. 4).

The candidate state indicators 320 are determined in advance on thebasis of a monitoring of the operation and microscopy images. In theillustrated example, a plurality of regions of interest 310, inparticular, are determined for the current state of the operation andthe candidate state indicators 320 are ascertained on the basis thereof.To this end, a machine-learned method could be used, for example. By wayof example, the candidate state indicators 320 may specify the qualityof the visibility of the corresponding regions of interest 310 (denotedby “ROI” in the figures) in the microscopy images. However, as a generalrule, other candidate state indicators 320 may also be determined, forexample independently of the regions of interest 310.

Then, there is a prioritization operation 391 as per Table 2: Example I.This is because one of the candidate state indicators 320 is prioritized(thick arrow in FIG. 6, “Feature 1”), i.e., filtering is carried out inthe illustrated case. This prioritization operation is carried out onthe basis of one or more context parameters 129.

The corresponding state indicator 320 is then transmitted as an input toa control model 330, which ascertains a candidate setting 340 for thesurgical microscope. This candidate setting 340 is then transmitted tothe surgical microscope as a final setting 125 (also referred to as“second setting”).

FIG. 7 illustrates aspects in relation to the control algorithm 130. Theexample in FIG. 7 corresponds, in principle, to the example in FIG. 6.In the example of FIG. 7, the control model 330 however determines twocandidate settings 340. Then, a prioritization is carried out amongthese candidate settings 340 within the scope of a furtherprioritization operation 392 in order to determine the setting 125, cf.Table 2: Example III.

FIG. 8 illustrates aspects in relation to the control algorithm 130. Theexample in FIG. 8 corresponds, in principle, to the example in FIG. 6.However, a plurality of candidate control models 330 are available inthe example of FIG. 8. One of the candidate control models 330 isprioritized within the scope of a further prioritization operation 393(“Model 1”; thick arrow) and used to determine the setting for thesurgical microscope, cf. Table 2: Example II.

FIG. 8 illustrates that the various candidate control models 330 areassociated with different state indicators. This may be the case becausethe candidate control models are associated with different assistancefunctionalities.

FIG. 9 illustrates aspects in relation to the control algorithm 130. Inthis case, only the prioritization operation 392 is used.

As a general rule, it is possible to flexibly combine the variousprioritization operations 391-393 with one another, or else use these onan individual basis.

FIG. 10 illustrates aspects in connection with the control algorithm130. In particular, FIG. 10 illustrates aspects in conjunction with thealgorithmic implementation of the control algorithm 130.

The control algorithm 130 comprises a plurality of sub-algorithms701-702, 711-712 in the example of FIG. 10. In this case, thesub-algorithms 701, 711 are associated with a first assistancefunctionality and the sub-algorithms 702, 712 are associated with asecond assistance functionality.

The sub-algorithm 701 and the sub-algorithm 702 are configured todetermine state indicators. To this end, the sub-algorithms 701-702 caneach resort to a monitoring of the operation, for example for currentregions of interest, and receive microscopy images of the surgicalmicroscope as an input. By way of example, the sub-algorithms 701-702can be machine-learned.

In this case it is possible in other examples for the determination ofthe state indicators to occur in advance of the control algorithm 130.

The sub-algorithms 701-702 then determine the state indicators and, onthe basis of the state indicators, the control models 711-712 can eachdetermine candidate settings. Target conflicts may arise in the process.

As a general rule, the control algorithm 130 could be implemented inmonolithic or modular fashion.

In the case of a monolithic implementation, various sub-algorithms701-702, 711-712 of the control algorithm 130 are implemented togetherand trained together in the case of machine-learned sub-algorithms. Byway of example, the sub-algorithms 701-702 which are configured todetermine the state indicators 320 can be implemented together. Thecontrol models 711-712 can also be implemented together.

In the case of a modular implementation different sub-algorithms701-702, 711-712 of the control algorithm 130 can be implemented in atleast partly separated fashion. In the case of machine-learnedsub-algorithms these can be trained separately. By way of example,different sub-algorithms can be associated with different assistancefunctionalities (assistance functionalities 751, 752 are illustrated inthe example of FIG. 10). By way of example, different sub-algorithms701-702 which are configured to determine the state indicators 320 couldbe used; these can be trained separately. Such an implementation isadvantageous in that in the case of changes in one or moresub-algorithms these can be replaced on an individual basis withoutinfluencing the training for other sub-algorithms.

An example for an application of the control algorithm 130 which isassociated with the two assistance functionalities 751-752 is describedbelow:

The assistance functionality 751 is configured to obtain an optimalhomogeneous illumination of the situs in the microscopy images from thesurgical microscope. Hence, the region of interest is the entire situs.On the basis of the microscopy images captured repeatedly during thesurgery, a mean brightness and a uniform distribution of the brightnessin the image are determined as state indicators, in each case by thesub-algorithm 701. By way of example, this could be implemented by ahistogram analysis. Then, on the basis of specified mapping ofbrightness to luminous intensity of the surgical microscope, the controlmodel 711 determines the required luminous intensity of the surgicalmicroscope 801 as a candidate for the setting.

The assistance functionality 752 is configured to avoid reflections inthe tissue around the tool tip. In this case, the region of interest isa region defined 2 cm around the tool tip. In this case, thecorresponding tools, i.e., surgical instruments, can be identified inimage-based fashion by the application of machine learning, for instanceby the sub-algorithm 702. Then, the number of reflections in the regionof interest is determined by the sub-algorithm 702 as a state indicator.To this end, the sub-algorithm could carry out a histogram analysisand/or machine learning. The control model 752 can then determine therequired luminous intensity as a candidate for the setting—on the basisof specified mapping between the number of reflections and the luminousintensity of the surgical microscope 801. A target conflict may arisefrom the two candidates for the setting: The assistance functionality751 states “brighter” for better light while the assistancefunctionality 752 states “darker” for fewer reflections.

Then it would be possible to prioritize between the two candidatesettings on the basis of a context parameter; cf. Table 2, Example III.In this example, the context parameter could describe an operationphase; cf. Table 1, Example IV.

A first operation phase could relate to working toward the aneurysm: inthis case, the surgeon would for example prefer a good illumination evenin the peripheral regions of the situs (maintaining overview) and, inreturn, accept reflections in their specific working region. A secondoperation phase relates to aneurysm clipping: in this case, the surgeonwould prefer, for example, to have fewer interfering reflections in theimage and, in return, accept a slightly worse peripheral illumination.

In the case of the modular solution the assistance functionalities 751,752 of the example above could be developed and tested independentlysince the context knowledge and the learned prioritization is onlyincluded between the steps of “models” and “parameter changecandidates”. The context extraction could also be learned independentlyof the assistance functionalities.

Changes to the assistance functionalities 751, 752 might also beundertaken since the prioritization already occurs at an earlier step inthe pipeline. Nevertheless, essential components of the respectiveassistance functionality 751, 752 can continue to be adopted in thiscase.

A further example for the aforementioned techniques is described below.

A setting of the surgical microscope 801 (also referred to as “secondsetting” above) is determined on the basis of a control algorithm 130.Here, the control algorithm 130 comprises a control model (cf. FIG. 10,control models 711, 712) which is designed as a machine-learnedalgorithm, for example as an artificial neural network. The controlmodel was trained in a training phase, for example on the basis of thefollowing state indicators: (i) relative position of the region ofinterest in the field of view of the microscope (viewing angle of thecameras); concealment of regions of interest by surgical instruments ortissue in the situs. The corresponding adjusted setting of the surgicalmicroscope 801 can be specified as “ground truth”.

In this case, graduated values for the state indicators (e.g., howprominent the concealment is) can be determined continuously while themicroscope is in use (e.g., on the basis of image processing).

Furthermore, it is however also conceivable to use state indicatorswhich indicate—for instance on the basis of a course of the operation—aprediction in conjunction with a corresponding feature for a futurepoint in time (e.g., a shortly anticipated degree of concealment). Byway of example, this can be solved by neural networks which were trainedin advance using real data.

During runtime, there is a situation-dependent prioritization of thestate indicators in real time (cf. Table 2, Example I): It is possibleto assume that surgeons reposition the microscope for one or morereasons. A reason for repositioning the microscope can consist of theview being impaired too much as a result of concealment. Or because anoptimal illumination is no longer ensured. However, at the same time thesurgeon also attempts to optimize other targets, e.g., distribution ofthe regions of interest in the image plane. Similar to the behaviour ofthe real surgeon, an attempt is made by way of the prioritization toprioritize those one or more state indicators which are indicative for avisual restriction.

To carry out this prioritization operation, the control algorithmincludes the current, the past and the anticipated future context. Thismeans that one or more context parameters are taken into account. Tothis end, use is made of machine learning-based methods which weretrained on real data, for example.

It is conceivable for context parameters which describe a past state ofthe operation to be processed, e.g. by intelligent filters whichdistinguish between relevant events and irrelevant events. Furthermore,it is conceivable that neural networks with inputs (i.e., current andpast context signals) make long-term or short-term context predictionswhich are included in the prioritization.

Thus, in summary, techniques were described above which describe aparameterization—for example a prioritization—of a control algorithm fora surgical microscope depending on a context of the operation. By way ofexample, a prioritization algorithm can be used to this end, the latterimplementing one or more prioritization operations on the basis of oneor more context parameters. Then, the surgical microscope can becontrolled on the basis of a corresponding setting which is obtainedfrom the control algorithm. By way of example, corresponding algorithms,i.e., the control algorithm and/or the prioritization algorithm, couldbe implemented on a suitable data processing unit.

In this case, the parameterization can be implemented in real time. Oneor more context parameters can be determined in real time, for exampleby means of an appropriate, possibly machine-learned algorithm. One ormore state indicators can be determined in real time, for example by amachine-learned algorithm (cf. FIG. 10, sub-algorithms 701, 702, or elsein advance of the control algorithm).

The operation can be monitored in real time. By way of example, one ormore regions of interest for the operation could be determined in realtime. A machine-learned method can be used to monitor the operation. Byway of example, a machine-learned method could be used to determineregions of interest. A model for the operation can be taken intoaccount. Monitoring the operation, for example in particular determiningregions of interest, can be implemented continuously.

One or more control models which are suitable for determining a settingfor the surgical microscope can be carried out in real time (cf. FIG.10, sub-algorithms 711, 712). To this end, machine-learned methods canbe used.

Rules-based methods can also be used as an alternative or in addition tomachine-learned methods. Other examples relate to self-learning orlearning methods, or behaviour-based methods.

It goes without saying that the features of the embodiments and aspectsof the disclosure described above can be combined with one another. Inparticular, the features can be used not only in the combinationsdescribed but also in other combinations or on their own withoutdeparting from the scope of the disclosure.

By way of example, techniques were described above, in which a settingfor a surgical microscope is determined on the basis of one or morestate indicators. In this case, variants would be conceivable in which asetting is also ascertained for different surgical devices usingcorresponding techniques, i.e., a parameterization of the controlalgorithm on the basis of one or more context parameters.

1. A method for controlling a surgical microscope during an operation ona patient, the method comprising: on the basis of a monitoring of theoperation and further on the basis of microscopy images from thesurgical microscope captured with at least one first setting of thesurgical microscope, determining one or more state indicators which areassociated with the at least one first setting of the surgicalmicroscope, carrying out a parameterization of a control algorithm ofthe surgical microscope on the basis of one or more context parametersof the operation, and on the basis of the parameterization, applying thecontrol algorithm to the one or more state indicators in order to thusdetermine a second setting of the surgical microscope.
 2. The method ofclaim 1, wherein the parameterization of the control algorithm comprisescarrying out at least one prioritization operation selected from: aprioritization within candidate state indicators, which are determinedusing different metrics on the basis of the microscopy images of thesurgical microscope; a prioritization within candidate control modelswhich are used by the control algorithm to ascertain settings of thesurgical microscope on the basis of the one or more state indicators;and/or a prioritization within candidate settings of the surgicalmicroscope which are obtained by at least one control model of thecandidate control models.
 3. The method of claim 2, wherein the at leastone prioritization operation is carried out to resolve at least onetarget conflict, wherein the at least one target conflict ischaracterized by different settings of the surgical microscope dependingon the result of the prioritization operation.
 4. The method of claim 1,wherein the one or more context parameters comprise one or more of thefollowing: a surgical device, for instance a surgical tool; a region ofinterest of the operation; a user stipulation; a phase of the operation;a type of operation; activities of the operation; planning data for theoperation; and/or at least one surgeon involved in the operation.
 5. Themethod of claim 1, wherein the one or more context parameters comprise aprediction for a future progress of the operation.
 6. The method ofclaim 1, further comprising: determining at least one context parameterof the one or more context parameters on the basis of a monitoring ofthe behaviour of a surgeon involved in the operation.
 7. The method ofclaim 1, wherein the monitoring of the operation relates to a numberand/or arrangement of one or more regions of interest of the operationwithin the patient.
 8. The method of claim 1, wherein the monitoring ofthe operation is implemented on the basis of a specified model of theoperation, which optionally comprises user-specific variants.
 9. Themethod of claim 1, wherein the parameterization is carried out on thebasis of a machine-learned algorithm which is optionally trained byreinforcement learning on the basis of feedback, which comprises adifference between the second setting of the surgical microscopedetermined by the control algorithm and a manually selected setting ofthe surgical microscope.
 10. The method of claim 1, wherein the one ormore state indicators are determined on the basis of one or more metricswhich are applied to the microscopy images, wherein the one or moremetrics evaluate one or more of the following properties of themicroscopy images: an image quality; a semantic content of themicroscopy images; and/or a visibility of regions of interest in themicroscopy images.
 11. The method of claim 10, wherein the one or morestate indicators comprise a plurality of state indicators which areassociated with different assistance functionalities and which aredetermined on the basis of different metrics.
 12. The method of claim11, wherein the different metrics assess the same properties of themicroscopy images contrariwise.
 13. The method of claim 1, wherein themonitoring of the operation relates to at least one of the following: acourse of the operation; an actual state of the operation; and/or aprogress of the operation in relation to a target state.
 14. The methodof claim 1, wherein the control algorithm comprises a plurality ofcontrol models, each of which is configured to determine a candidatesetting of the surgical microscope on the basis of at least onecorresponding state indicator of the one or more state indicators,wherein the plurality of control models each comprise a machine-learnedalgorithm, wherein the machine-learned algorithms of the plurality ofcontrol models are trained separately.
 15. A device comprising aprocessor configured to carry out operations comprising: on the basis ofa monitoring of an operation and further on the basis of microscopyimages from a surgical microscope captured with at least one firstsetting of the surgical microscope, determining one or more stateindicators which are associated with the at least one first setting ofthe surgical microscope, carrying out a parameterization of a controlalgorithm of the surgical microscope on the basis of one or more contextparameters of the operation, and on the basis of the parameterization,applying the control algorithm to the one or more state indicators inorder to thus determine a second setting of the surgical microscope. 16.The device of claim 15, wherein the parameterization of the controlalgorithm comprises carrying out at least one prioritization operationselected from: a prioritization within candidate state indicators, whichare determined using different metrics on the basis of the microscopyimages of the surgical microscope; a prioritization within candidatecontrol models which are used by the control algorithm to ascertainsettings of the surgical microscope on the basis of the one or morestate indicators; and/or a prioritization within candidate settings ofthe surgical microscope which are obtained by at least one control modelof the candidate control models.
 17. The device of claim 16, wherein theat least one prioritization operation is carried out to resolve at leastone target conflict, wherein the at least one target conflict ischaracterized by different settings of the surgical microscope dependingon the result of the prioritization operation.
 18. The device of claim15, wherein the one or more context parameters comprise one or more ofthe following: a surgical device, for instance a surgical tool; a regionof interest of the operation; a user stipulation; a phase of theoperation; a type of operation; activities of the operation; planningdata for the operation; and/or at least one surgeon involved in theoperation.
 19. The device of claim 15, wherein the one or more contextparameters comprise a prediction for a future progress of the operation.20. The device of claim 15, wherein the operations furthermore comprise:determining at least one context parameter of the one or more contextparameters on the basis of a monitoring of the behaviour of a surgeoninvolved in the operation.